Overview

This PR introduces configuration for PyTorch Inductor backend,
allowing users to select predefined config templates that set groups of
torch._inductor.config overrides. This provides an extension to PyTorch's
official "mode" concept while maintaining full compatibility with existing
test_compiler.py framework.

Motivation

Previously, InductorBackend accepted only basic config parameters through
individual inductor_config dictionary entries. Users could not easily enable
common combinations of Inductor options such as:

• CUTLASS-based GEMM kernels - For optimal GEMM performance on modern GPUs
• CUDA Graphs - To reduce kernel launch overhead for small batch inference
• Model freezing - To inline weights as constants for deployment optimization
• TMA (Tensor Memory Accelerator) - For H100+ GPUs with hardware acceleration

This PR addresses these limitations by introducing config templates - pre-defined,
well-tested combinations of torch._inductor.config options that users can select
by name. Templates, mode, and options are mutually exclusive, providing clear
separation of concerns.

Changes Summary

1. Inductor Backend Configuration Templates

File: graph_net_bench/torch/backend/inductor_backend.py

Features

• _INDUCTOR_CONFIG_TEMPLATES dictionary with 6 predefined templates
• Parameter: graph_net_inductor_config_template - select template by name
• Mutual exclusivity: exactly one of template/mode/options can be specified
• No global state mutation: uses torch.compile's options parameter directly

Supported Templates

Note that the TMA template works universally across GPU architectures:

• H100+ (CC >= 9.0): Enables TMA persistent kernels
• A100 or other GPUs: Enables non-TMA persistent kernels as fallback

No runtime error occurs on GPUs without TMA support.

2. CUDA Graphs Compatibility Fix

File: graph_net_bench/torch/test_compiler.py

When CUDA Graphs is enabled, output tensor pointers are recorded to CUDA Graph buffers.
Subsequent model calls overwrite these buffers, causing errors when accessing compiled
output after eager run. Fixed by cloning outputs immediately:

if isinstance(outs, torch.Tensor):
    outs = outs.clone()
elif isinstance(outs, tuple):
    outs = tuple(t.clone() if isinstance(t, torch.Tensor) else t for t in outs)

Note: eval_backend_perf.py and eval_backend_diff.py are unaffected
(torch.save/torch.load creates independent copies).

3. Test

File: test/inductor_backend_test.py (new file, ~290 lines)

Test Coverage:

• Template validation: 11 tests
• Parameter handling: 5 tests
• Config validation: 9 tests
• Integration: 1 test
• Total: 26 tests

Usage

# Schema: exactly one of template/mode/options can be specified
--config '{"graph_net_inductor_config_template": "<template_name>"}'    # template
--config '{"mode": "<mode_name>"}'                                    # mode
--config '{"options": {...}}'                                          # custom

# Example: use max-autotune template
python -m graph_net_bench.torch.test_compiler \
    --compiler inductor \
    --model-path samples/torchvision/alexnet \
    --config "$(echo '{"graph_net_inductor_config_template": "max_autotune"}' | base64 -w0)" \
    --trials 5 --warmup 3

Documentation References

All configuration keys verified against PyTorch source code:

• Config File: https://github.com/pytorch/pytorch/blob/main/torch/_inductor/config.py
• Compile API: https://pytorch.org/docs/stable/generated/torch.compile.html